Experiences on enhancing data collection in large networks

We improve and validate TICP [3], our TCPfriendly reliable transport protocol to collect information from a large number of Internet entities. A collector machine sends probes to a set of information sources that reply by sending back their reports. TICP adapts the sending rate of probes in a way similar to TCP for the purpose of avoiding network congestion and implosion at the collector. Lost reports are requested again by TICP until they are correctly received by the collector. In a first part of this work, we add to TICP a mechanism to cluster information sources in order to probe sources behind the same bottleneck together. This ensures a smooth variation of network conditions during the collection session and hence, an efficient handling of congestion at network bottlenecks. We run simulations in ns-2 over realistic topologies to compare TICP before and after clustering. We also implement the protocol in C++ and test it over the PlanetLab platform. All experiments prove the outperformance of TICP over non adaptive solutions and the interest of the clustering mechanism in shortening the duration of the collection session and in decreasing the ratio of lost packets. In a second part, we adapt TICP to collect large amounts of information from each data source. By the means of simulations, we compare the performance obtained by TICP to that obtained when the information maintained by the different sources is collected by parallel TCP connections. Again, the simulations show that TICP yields shorter collection sessions due to its inherent multiplexing capability. Finally, in a last part, we study the impact of delegating collection to some proxy sources that collect from other sources on behalf of the collector and that the collector probes later to get their collected data. We explain our method to choose the proxy collectors and we show by simulations that for a judicious choice of proxy collectors, one can decrease considerably the collection session duration.